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Abstract-This  study  focused  on  man-machine  interface  of  FES 
system  for  restoring  gait  of  hemiplegic  patients.  A  method  of 
recognition  of  lower  limb  movements  using  an  artificial  neural 
network  (ANN)  was  examined  in  monitoring  restored  motions 
and  in  giving  control  command  with  normal  subjects  and  a 
hemiplegic  patient.  Acceleration  signals  were  measured  with  a 
three-axis  accelerometer  attached  to  the  heel  of  the  normal  side 
(right  side)  during  walking  for  using  in  the  recognition.  Subjects 
performed  some  specific  movements  by  their  normal  lower 
limbs  supposing  control  command  input  in  the  walking 
measurements.  The  ANN  recognized  three  different  walking 
patterns,  which  were  level  floor  walking,  going  up  and  down 
stairs,  based  on  the  acceleration  waveforms  with  about  80%  of 
recognition  rate  for  normal  subjects  and  above  70%  for  the 
patient.  A  similar  structure  of  the  ANN  discriminated  four 
specific  movements  by  the  lower  extremity  with  more  than  90% 
of  recognition  rate  after  the  third  performance  of  the  movement 
simulated  by  using  recognition  and  mis-recognition  rates  for 
experimentally  measured  data.  The  method  was  found  to  be 
useful  in  monitoring  FES  movements  for  safety  and  in  giving 
control  commands  to  the  FES  system  without  using  upper  limbs. 
Keywords  -  FES,  neural  network,  gait,  hemiplegics 

I.  Introduction 

Walking  of  hemiplegic  patients  on  level  floor  has  been 
improved  by  controlling  ankle  dorsiflexion  using  functional 
electrical  stimulation  (FES).  The  main  purpose  of  this 
method  is  to  correct  the  drop  foot  during  walking.  The 
method  usually  uses  foot  switches  to  detect  the  toe  off  and 
the  heal  contact  of  the  paralyzed  foot.  Electrical  stimulus 
pulse  train  is  applied  to  muscles,  mainly  the  tibialis  anterior, 
on  the  basis  of  predetermined  stimulation  data  during  the 
swing  phase  detected  by  the  foot  switches. 

It  is  required  to  make  secure  the  patient’s  safety  during 
FES  gate  for  practical  use.  From  this  point  of  view, 
monitoring  restored  motions  can  be  an  important  function  of 
FES  system  in  addition  to  feedback  control.  Although  non¬ 
restraint  measurement  of  walking  [1]  and  detection  of  gait 
phase  [2]  have  been  studied,  detection  of  walking  pattern  at 
every  step  has  not  been  realized. 

For  going  up  and  down  stairs  freely  in  addition  to  level 
floor  walking,  it  is  necessary  to  control  a  lot  of  muscles 
relating  to  the  ankle,  the  knee  and  the  hip  joints  using 
appropriate  stimulation  data  to  restore  different  walking 
patterns.  That  is,  a  lot  of  control  commands  are  required  to 
restore  many  movements.  In  restoring  motor  functions  of 


lower  extremity,  a  patient  also  uses  usually  crutch,  cane,  stick 
and  so  on.  In  this  case  it  is  not  desirable  to  use  upper 
extremities  for  giving  control  commands  because  of  assuring 
the  safety  of  patients.  Alternatives  to  using  upper  extremities 
for  the  command  input  may  be  to  use  head,  shoulder  or  lower 
limb  of  the  normal  side. 

We  have  shown  that  an  artificial  neural  network  (ANN) 
had  the  ability  to  recognize  a  specific  movement  measured 
with  the  three-dimensional  position  sensor  [3].  Using  this 
technique,  we  focused  on  developing  man-machine  interface 
of  FES  system  for  monitoring  restored  motions  and  giving 
control  commands.  In  this  paper,  the  ability  of  an  ANN  was 
examined  in  recognition  of  walking  patterns  based  on 
acceleration  waveforms  measured  with  a  3-axis 
accelerometer  during  walking  on  the  level  floor,  going  up  and 
down  stairs  with  neurologically  intact  subjects  and  a 
hemiplegic  patient.  The  ability  of  recognition  of  some 
specific  movements  for  giving  control  commands  was  also 
examined  using  the  similar  structure  of  the  ANN. 

II.  Methodology 

A.  Measurement  of  lower  movements  of  the  normal  side 

Subjects  were  5  neurologically  intact  male  (22-24yrs.) 
and  a  hemiplegic  patient  (female,  55yrs.).  Three-axis 
accelerometer  (8692C50M1,  KISTLER)  was  attached  to  the 
heel  of  the  shoe  of  normal  side  (Fig.l).  The  right  side  was 
assumed  to  be  the  normal  side  in  the  case  of  normal  subjects. 
Foot-switches  were  attached  on  the  heel  and  the  thumb  pad  of 
the  foot  of  the  normal  side  in  order  to  detect  the  stance  and 
the  swing  phases  for  ANN  learning.  Both  acceleration  and 
foot-switch  data  were  recorded  with  data  recorder  (RD-135T, 
TEAC).  These  data  were  low  pass  filtered  (fc= 20Hz)  and 
sampled  at  200Hz  into  personal  computer  after  measurement. 

Walking  patterns  for  measurement  were  i)  straight 
walking  on  the  level  floor,  ii)  going  up  stairs  and  iii)  going 
down  stairs.  In  each  walking  measurement,  subjects 
performed  specific  movements  by  their  normal  side  for 
supposed  control  command  input  at  1)  the  start  of  level  floor 
walking,  2)  the  start  of  going  up  stairs,  3)  the  start  of  going 
down  stairs  and  4)  the  stop  of  walking.  Specific  movements 
were  #1)  knee  flexion  for  command  1),  #2)  upward 
movement  of  the  knee  for  command  2),  #3)  inner  rotation  of 
the  lower  thigh  with  toe  contact  for  command  3)  and  #4) 
plantar-flexion  of  the  ankle  with  toe  contact  for  command  4). 
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Fig.  1  Arrangement  of  an  accelerometer  and  foot  switches  for  the 
right  foot.  Directions  of  the  3-axis  accelerometer  are  shown  in 
the  figure. 
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Fig.2  Structure  of  the  artificial  neural  network  (ANN)  used  for 
recognition  of  walking  patterns.  The  value  of  n  was  20. 


The  first  step  was  made  by  the  normal  side  in  the  level 
floor  walking  and  going  up  stairs.  In  the  case  of  going  down 
stairs,  the  paralyzed  side  made  the  first  step.  The  hemiplegic 
patient  used  the  stick  during  level  floor  walking  and  the  grab 
rail  during  stair  walking.  There  is  no  other  restriction  for 
subjects  during  walking.  The  numbers  of  measurements  were 
10  for  three  walking  patterns  with  normal  subjects,  10  for 
level  floor  walking  and  5  for  stair  walking  patterns  with  the 
patient.  In  the  case  of  stair  walking  of  normal  subjects,  one  or 
two  steps  of  the  normal  side  on  the  level  floor  were  included 
before  and  after  the  stair  walking.  In  order  to  lightening  a 
burden  on  the  patient  for  the  measurement,  we  instructed  her 
to  stop  stair  walking  halfway. 


were  accumulated  and  added  to  connection  weights  after  all 
the  learning  data  were  given.  The  adjustment  of  weights  was 
repeated  until  the  squared  error  function  was  less  than 
reference  value.  The  maximum  repetition  number  of  the 
learning  was  40,000.  Learning  coefficient  was  0.02.  Initial 
value  of  coefficient  of  the  inertia  term  was  0,  the  increment 
of  the  coefficient  was  0.01  and  the  maximum  was  0.9. 

Each  neuron  of  the  output  layer  made  numerical  value 
between  0  and  1 .0  using  sigmoidal  function.  The  output  value 
that  was  larger  than  or  equal  to  0.5  meant  that  the  neuron 
detected  the  movement.  When  detection  results  of  all  output 
neurons  were  correct,  the  ANN  made  correct  recognition. 


B.  Recognition  of  walking  patterns  for  monitoring 

Level  floor  walking,  going  up  and  going  down  stairs  were 
recognized  by  the  three-layer  feed-forward  style  ANN  shown 
in  Fig.2.  The  numbers  of  neurons  were  60  for  the  input  layer, 
10  for  the  hidden  layer  and  3  for  the  output  layer.  Measured 
acceleration  signals  were  low  pass  filtered  (/c=3Hz)  and 
sampled  at  20Hz  for  the  input  to  neurons  of  the  input  layer. 
The  number  of  the  neuron  of  the  input  layer  was  60  (20  for 
each  component)  because  the  swing  phase  was  about  Is. 
Changes  of  the  acceleration  signal  from  the  amplitude  of  1~ 
20  samples  before  were  given  to  each  neuron  of  the  input 
layer  for  the  period  of  Is  before  the  recognition  (see  Fig.2). 
Each  neuron  of  the  output  layer  outputs  the  recognition  result 
of  level  floor  walking,  going  up  or  going  down  stairs, 
respectively. 

Error  back  propagation  algorithm  was  used  for  learning  of 
the  ANN.  Teacher  signal  F1IGH  (numerical  value  was  0.99) 
was  applied  at  the  end  of  the  swing  phase  detected  by  the  foot 
switches  i)  between  the  second  step  and  the  step  before  the 
stop  for  level  floor  walking,  ii)  between  the  second  step  from 
the  bottom  of  stairs  and  the  top  step  for  going  up  stairs  and 
iii)  between  the  second  step  from  the  top  and  the  first  step  on 
level  floor  just  after  the  stair  walking  for  going  down  stairs. 
Teacher  signal  LOW  (numerical  value  was  0.01)  was  given  at 
other  time.  Amounts  of  adjustment  of  connection  weights 
were  5  times  of  the  calculated  value  at  when  teacher  signal 
was  FlIGH  (0.99),  while  those  at  when  teacher  signal  was 
LOW  (0.01)  were  the  calculated  values.  The  adjustments 


C.  Recognition  of  specific  movements  for  control  commands 

A  method  of  recognizing  specific  movements  was  studied 
for  using  in  control  command  input.  Structure  of  the  ANN 
used  in  specific  movement  recognition  was  similar  to  the 
ANN  for  walking  pattern  recognition.  The  difference  was  the 
number  of  neurons  in  the  output  layer.  Four  neurons  were 
used  in  the  output  layer  for  specific  movement  recognition. 
Each  neuron  detected  each  movement  described  in  section  II 
A.  Signal  conditioning  for  the  input  to  the  ANN  was  the  same 
as  for  the  ANN  for  walking  pattern  recognition. 

Two  measured  data  for  each  walking  were  used  for  the 
ANN  learning.  Acceleration  signals  during  movement  #2  and 
#3  in  the  learning  data  were  copied  and  added  to  the  end  of 
the  learning  data  three  times.  Teacher  signal  HIGH 
(numerical  value  was  0.99)  was  applied  at  the  end  of  specific 
movement  and  LOW  (numerical  value  was  0.01)  was  applied 
at  other  time.  The  other  parts  of  learning  method  were  the 
same  as  that  for  walking  pattern  recognition.  Output  value 
also  had  same  meaning  as  that  of  walking  pattern  recognition. 

III.  Results 

An  example  of  measured  acceleration  signal  during  level 
floor  walking  is  shown  in  Fig.3.  The  subject  performed 
specific  movements  at  the  beginning  of  walking  (movement 
#1)  and  just  after  the  final  step  of  walking  (movement  #4)  in 
this  walking.  From  the  second  step  of  the  normal  side  to  the 
step  before  the  final  one  were  used  for  the  learning  and  the 
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recognition. 

A.  Recognition  of  walking  patterns 

Recognition  rate  was  defined  as  the  ratio  of  the  number  of 
correct  recognition  to  the  total  number  of  steps  and 
movements  for  recognition  [%].  Recognition  rates  to 
unlearned  data  (7  measured  data  for  normal  subjects,  3  data 
for  the  patient)  are  shown  in  Table  1.  Mean  recognition  rates 
were  about  80%  for  normal  subjects  and  about  more  than 
70%  for  the  patient. 

With  the  6  subjects  including  the  patient,  the  mis- 
recognitions  that  output  numerical  value  was  less  than  0.5 
and  that  different  movement  was  detected  at  when  teacher 
signal  was  HIGH  (0.99)  were  61.1%  and  2.6%  of  total 


number  of  mis-recognition  for  level  floor  walking,  61.5%  and 
3.7%  for  going  up  stairs  and  67.2%  and  6.0%  for  going  down 
stairs,  respectively.  The  other  mis-recognition  (36.4%  of  total 
number  of  mis-recognition  for  level  floor  walking,  34.8%  for 
going  up  stairs  and  26.8%  for  going  down  stairs)  was  the  case 
of  the  output  numerical  value  was  larger  than  or  equal  to  0.5 
at  when  teacher  signal  was  LOW  (0.01). 

B.  Recognition  of  specific  movements 

Table  2  shows  the  recognition  results  of  specific 
movements  for  unlearned  data.  The  recognition  rate  was 
defined  as  the  ratio  of  the  number  of  correct  recognition  to 
the  total  number  of  performed  movement.  The  numbers  of 
unlearned  data  were  24  for  normal  subjects  and  14  for  the 
patient. 

Recognition  rate  to  four  specific  movements  was  over 
70%  for  normal  subjects.  Most  of  mis-recognitions  were 
caused  by  low  output  values  (less  than  0.5)  at  when  teacher 
signal  was  HIGH.  In  this  case,  the  ANN  did  not  detect  a 
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movement  and  a  step.  It  was  66.0%  of  the  total  number  of 
mis-recognition  in  average  for  the  normal  subjects.  Mis- 
recognition  caused  by  detecting  a  different  movement  at 
when  teacher  signal  was  HIGH  (0.99)  and  detecting  a 
movement  at  when  teacher  signal  was  LOW  (0.01)  were 
21.7%  and  12.3%  of  the  total  number  of  mis-recognition  in 
average  for  the  normal  subjects,  respectively. 

Supposing  re-input  of  control  command  in  the  case  of 
output  value  of  the  ANN  was  less  than  0.5  when  a  specific 
movement  was  performed,  recognition  rates  Pi,  P2  and  P3 
were  calculated  by  following  equations: 

Pi  =  s  :  first  movement 

P2  =  s+rs  :  second  movement 

P3  =  s+rs+r2s  :  third  movement 
where  .?  is  the  recognition  rate  for  the  first  movement.  Mean 
recognition  rate  of  the  four  movements  was  used  as  the  value 
of.?  for  each  subject.  The  value  of  r  is  the  rate  of  the  number 
of  mis-recognition,  which  was  caused  by  low  output  value 
(less  than  0.5)  at  when  teacher  signal  was  HIGH  (0.99),  to  the 
total  number  of  recognition.  Results  are  shown  in  Table  3. 
After  the  supposed  third  input  of  control  command,  the 
system  was  expected  to  provide  92.4%  of  recognition  rate  in 
average  with  normal  subjects. 

IV.  Discussion 

In  walking  pattern  recognition,  most  of  the  mis- 
recognition  at  when  teacher  signal  was  LOW  (0.01)  was 
caused  by  specific  movements.  The  ANN  mis-recognized 
specific  movements  for  giving  control  command  as  one  of 
three  walking  patterns.  Especially,  movement  #2  for  the  start 
of  going  up  stairs  was  incorrectly  recognized  as  walking 
pattern  of  going  up  stairs.  Therefore,  performing  specific 
movements  clearly  is  expected  to  decrease  the  mis- 
recognition  rate.  In  the  case  of  normal  subjects,  it  is  possible 
to  obtain  higher  recognition  rate  for  specific  movements  than 
those  of  experimental  results  by  performing  the  movements 
more  clearly.  Selecting  specific  movements,  which  is 
different  from  walking  patterns,  also  can  be  a  solution  for 
decreasing  the  mis-recognition. 

However,  variation  of  acceleration  pattern  during  walking 
was  sometimes  observed  in  recorded  signals  causing  low 
recognition  rate,  e.g.  in  the  case  of  level  floor  walking  with 
subject  B  and  going  down  stairs  with  subject  E.  Variation  of 
walking  speed  also  made  difference  in  acceleration  pattern 
between  measurements.  Mis-recognition  caused  by  variation 
of  acceleration  pattern  in  the  same  walking  has  to  be  solved 
for  practical  application. 

For  specific  movement  recognition,  the  average 
recognition  rate  after  the  supposed  third  command  input  was 
high  (more  then  90%)  with  the  normal  subjects.  This  shows 
that  the  method  can  be  practical.  In  the  case  of  the  patient, 
however,  the  rate  was  low.  A  possible  reason  for  the  low  rate 
was  the  difficulty  of  performing  specific  movements  by  the 
normal  side  with  the  support  of  the  paralyzed  side.  Even  if  it 
was  the  reason,  high  recognition  rate  for  the  movement  #4 
was  obtained  (Table  2).  The  movement  #4  was  plantar- 
flexion  of  the  ankle  with  toe  contact.  The  toe  contact  of  the 


Table  3  Measured  (Pi)  and  simulated  ( P2  and  P3)  mean  recognition 
rates  for  specific  movements  [%].  P \  is  the  measured  ratio.  Pi  and 
P3  mean  simulated  rates  for  supposed  second  and  third  movement, 
respectively. 


subject 

P\ 

Pi 

Pi 

normal  A 

70.0 

84.0 

86.8 

B 

88.8 

93.2 

93.4 

C 

73.8 

89.4 

92.8 

D 

73.8 

92.2 

96.8 

E 

77.5 

90.1 

92.1 

mean 

76.8 

89.8 

92.4 

patient 

41.0 

54.1 

58.3 

normal  side  is  considered  to  improve  the  stability  during 
performing  specific  movements  by  the  normal  side.  This 
indicates  that  selecting  movements  carefully  is  necessary  to 
obtain  high  recognition  rate. 

V.  Conclusion 

This  study  examined  the  use  of  ANN  for  recognition  of 
walking  patterns  and  specific  movements.  Movements  were 
measured  with  a  three-axis  accelerometer  attached  to  the  heel 
of  shoe  of  the  normal  side.  The  technique  was  found  to  be 
useful  for  monitoring  walking  and  giving  control  command. 
The  technique  is  expected  to  provide  information  for  assuring 
the  safety  of  patients  and  to  improve  operationality  of  the 
FES  system  for  practical  use. 
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